# Linear Attention

The Linglong Linear Preview is a hybrid linear sparse large language model open-sourced by InclusionAI, with a total of 17.1B parameters and 3.0B activated parameters. This model implements long-text reasoning based on a hybrid linear attention mechanism, achieving near-linear computational complexity and near-constant space complexity during inference.

MoLFormer is a chemical language model pre-trained on 1.1 billion molecular SMILES strings from ZINC and PubChem. This version uses 10% samples from each dataset for training.